Run positioned saves at READ COMMITTED on MySQL - #86

For 0.5.0, as this contains a breaking change. Replaces #72.

Problem

Under MySQL's default REPEATABLE READ, a transaction's first plain read fixes the snapshot that all of its later plain reads see. Locking reads and UPDATEs see the latest data, but MAX(position), position_was and the relative position lookups don't.

So anything that reads before the scope lock hides changes committed while we waited for it: the gem loading the belongs_to association in order to lock it, a validation, a callback, or nested attributes. The result is duplicate positions, RecordNotUnique where there's a unique index, or a new record placed in the wrong spot. PostgreSQL's default READ COMMITTED takes a fresh snapshot for each statement, so it isn't affected.

Change

There are two commits.

Lock scopes by key. The lock no longer loads the associated record first. It locks by primary key using the foreign keys in memory, selecting only ids and ignoring default scopes. This covers polymorphic and composite keys. Default scopes can no longer add preloads to a lock, or hide the record so that no lock is taken. Scopes without an association still lock whole rows: selecting only ids lets MySQL lock through a covering index on the position column, and a transaction queued there blocks our own writes into the gap before it, which deadlocks.

Run positioned saves at READ COMMITTED on MySQL. positioned includes a small module that handles the two ways a save can start:

• A save that begins its own transaction runs at READ COMMITTED, by overriding with_transaction_returning_status, which save, save!, destroy, destroy!, touch, update and update! all go through. The override opens the real transaction, so it re-raises ActiveRecord::Rollback for the one that joining swallows.
• A save that joins an open transaction can't change its level, so it locks as soon as it begins, before validations or callbacks can read. This matters more than it might seem, because Rails opens a transaction around association.create and <<.

Both are needed. Neither alone handles every row below.

Breaking changes

On MySQL only. PostgreSQL and SQLite are unchanged.

• Every save on a positioned model runs at READ COMMITTED, not just those that change a position, because update opens its transaction before we know what's changing. That includes its callbacks and any records it autosaves.
• FOR UPDATE no longer stops inserts into the gaps between rows in those saves. This is the risky one, as code relying on gap locks loses that protection with no error.
• An app deliberately running at a stricter level has these saves lowered, unless it chose the level with ActiveRecord.with_transaction_isolation_level, which takes precedence.
• Per the MySQL docs, writes at this level need binlog_format to be ROW (the default) or MIXED. This wasn't tested.
• Transactional tests hold a transaction open, so apps won't see any of this in their test suites.

To opt out:

ActiveRecord::Base.positioning_isolation = nil # For the whole app, in an initializer
self.positioning_isolation = nil               # For one model

Results by call pattern

MySQL on Rails 8.1. Each case has a validation that reads, with a concurrent create at position 1 straight after the first query. The first column is the last release.

Not covered

The last two rows. A snapshot taken before the save begins can't be undone: MySQL rejects SET TRANSACTION once a transaction is in progress, and the SESSION form doesn't affect the running one. The README says to open the transaction with transaction(isolation: :read_committed), and there's a test for that advice.

Alternative considered

Overriding transaction on positioned classes as well would also fix Item.transaction { read; item.update }. It would silently change the level of every Item.transaction block in an app, including multi-model ones, and without the early lock it wouldn't help List.transaction { item.update }. I left it out.

Testing

• 260 tests pass, three times each, on MySQL (Rails 7.2, 8.0, 8.1), PostgreSQL and SQLite (Rails 8.1).
• Most test classes wrap each test in a transaction, which makes the override do nothing. So I also ran the whole suite on MySQL with those wrappers removed: all 260 pass, with 1,184 transactions run at READ COMMITTED.
• Six mutations each make the expected tests fail: not including the module, not re-raising the rollback, no early lock, ignoring an app's chosen level, not checking for an open transaction, and ignoring the opt-out.
• In a real Rails 8.1 and MySQL app, its 733 tests pass and a nested attributes race that silently corrupted a list on 0.4.9 now gives the right result.

Query counts in that app, which has default scopes that preload attachments and rich text:

These exclude BEGIN, COMMIT and SET TRANSACTION, so a save at READ COMMITTED has one extra round trip on top.

Not tested: MariaDB, Trilogy, and proxies such as ProxySQL or Vitess, which need SET TRANSACTION and BEGIN to reach the same backend connection.

🤖 Generated with Claude Code